Enhancing Credit Card Fraud Detection Using Random Forest and SMOTE

Abstract

Credit card theft costs the global banking system billions of dollars every year and erodes user confidence in electronic payments. Fraudulent transactions are rare (usually < 0.2 % of all records) and this rarity poses a basic problem for automated detection: a classifier trained on raw data learns to predict the majority class. The resulting model is surprisingly accurate in general, yet fails to detect much of the fraud it was created for. To directly address such imbalance, this research integrates a bespoke Random Forest classifier with the Synthetic Minority Over-sampling Technique (SMOTE). The pipeline is tested on the public European credit card transaction dataset with 284,807 transactions with just 492 (0.172%) fraud cases. The SMOTE-RF model therefore performs better than Logistic Regression, Decision Trees and the normal Random Forest without oversampling, with a precision of 0.947, a recall of 0.921, an F1-score of 0.934, a Matthews Correlation Coefficient (MCC) of 0.929 and an AUC-ROC of 0.983. An ablation research reveals that most of the recall gain is driven by SMOTE, whereas precision is mostly enhanced by tuning Random Forest hyperparameters. All these results indicate that SMOTE-RF is a feasible and interpretable solution for real-world fraud detection.

Introduction

This paper investigates the use of a hybrid SMOTE–Random Forest (SMOTE-RF) framework for detecting credit card fraud in highly imbalanced transaction datasets. As digital payments, online shopping, and mobile banking continue to expand, financial fraud has become a major global concern, with fraud losses exceeding billions of dollars annually. Traditional machine learning models often struggle because fraudulent transactions represent less than 0.2% of all transactions, causing classifiers to favor legitimate transactions and miss many fraud cases.

Objective

The study aims to improve fraud detection by combining:

• Synthetic Minority Over-sampling Technique to balance the dataset by generating synthetic fraud samples.
• Random Forest to classify transactions using an ensemble of decision trees.

The research contributions include:

1. Comparing SMOTE-RF with five baseline fraud detection models.
2. Conducting an ablation study to measure the individual effects of SMOTE, class weighting, and threshold tuning.
3. Analyzing precision–recall trade-offs for practical deployment.
4. Providing hyperparameter recommendations for fraud detection systems.

Literature Review

Earlier fraud detection systems relied on rule-based methods and statistical models, which suffered from high false-positive rates and limited adaptability.

Key findings from prior research include:

• Ensemble methods generally outperform single classifiers.
• Random oversampling may lead to overfitting.
• Undersampling can discard useful information.
• SMOTE generates synthetic minority samples through interpolation, reducing overfitting while improving class balance.
• Advanced variants such as Borderline-SMOTE, ADASYN, and SMOTE-ENN further refine minority class generation.
• Ensemble approaches such as Random Forest and XGBoost consistently perform well on fraud detection tasks.

Dataset and Preprocessing

The study uses the well-known European Credit Card Fraud Detection dataset from Université libre de Bruxelles.

Dataset characteristics:

• Total transactions: 284,807
• Fraudulent transactions: 492
• Fraud rate: 0.172%
• Class imbalance ratio: approximately 578:1
• Features:
  • 28 anonymized PCA components (V1–V28)
  • Time
  • Amount
  • Class label

Preprocessing steps:

1. Robust scaling of Time and Amount features.
2. Stratified 80/20 train-test split.
3. Application of SMOTE only to the training set to prevent data leakage.
4. No feature selection, allowing Random Forest to determine feature importance.

The final oversampling strategy produced a minority-to-majority ratio of 1:10.

Methodology

SMOTE

SMOTE addresses class imbalance by creating synthetic fraud examples between existing minority-class samples rather than duplicating them. Using k-nearest neighbors (k = 5), new fraud samples are generated through interpolation within the minority-class feature space.

Random Forest

The Random Forest classifier:

• Builds multiple decision trees using bootstrap sampling.
• Uses random feature selection at each split.
• Employs Gini impurity to determine optimal splits.
• Produces final predictions through majority voting across trees.

Advantages include:

• Strong resistance to overfitting.
• Effective handling of high-dimensional data.
• Built-in feature importance estimation.
• Reliable probability outputs for threshold tuning.

Optimized Hyperparameters

The best-performing configuration included:

• 300 trees
• Unlimited tree depth
• Balanced class weights
• Square-root feature sampling
• Decision threshold of 0.4

Hyperparameters were optimized using 5-fold stratified cross-validation.

Evaluation Metrics

Because accuracy is misleading on highly imbalanced datasets, the study focuses on:

• Precision
• Recall (Sensitivity)
• F1-score
• Matthews Correlation Coefficient (MCC)
• AUC-ROC
• AUC-PR (Precision–Recall Area Under Curve)

These metrics better reflect fraud detection effectiveness.

Results

Model Comparison

The proposed SMOTE-RF model achieved the best overall performance:

The SMOTE-RF model delivered the highest F1-score, MCC, AUC-ROC, and AUC-PR, indicating superior fraud detection performance.

Ablation Study

The study evaluated the contribution of each pipeline component:

Key observations:

• SMOTE provided the largest improvement in recall.
• Class weighting further improved fraud detection.
• Threshold tuning produced additional gains with minimal precision loss.
• All three components contributed significantly to final performance.

Conclusion

In this study, SMOTE was applied along with a modified Random Forest classifier to detect credit card fraud under high class imbalance. On the typical European credit card fraud benchmark, the SMOTE-RF pipeline surpassed five baselines, including Logistic Regression, Decision Trees, and Random Forest without oversampling with precision 0.947, recall 0.921, F1-score 0.934, MCC 0.929, and AUC-ROC 0.983. The ablation investigation shows that most of the memory enhancement comes from SMOTE oversampling with the tiny supplementary enhancements come from class weighting and threshold adjustment. For this dataset, the best trade-off between recall and precision was achieved using a minority-to-majority ratio of 1:10. The examination of feature importance shows that the most important fraud signals are V14, V17, V12, V10 and V11 which is in line with earlier studies. Overall, our results imply that SMOTE-RF is a practical and interpretable choice for fraud detection systems that will be used in production. It\'s a beneficial match in cases where regulators want some level of explainability due to its quick inference latency and inherent feature importance.

References

[1] B. Ottersten, A. Stojanovic, A. C. Bahnsen, and D. Aouada, (2016). Feature Engineering Techniques for Credit Card Fraud Detection. 51, 134-142 Expert Systems with Applications. [2] Jha, S., Westland, J. C., Bhattacharyya, S., & Tharakunnel, K. , (2011). A comparative study of credit card fraud data mining 50(3), 602-613; Decision Support Systems. [3] Blagus, R. and Lusa, L.. (2013). SMOTE for high dimensional unbalanced data. 1–16 in BMC Bioinformatics, 14(1) [4] Breiman, L. (2001). Random forests 45(1), 5-32; Machine Learning. [5] Bowyer, K. W., Hall, L. O., Chawla, N. V., and Kegelmeyer, W. P. (2002). SMOTE is Synthetic Minority Over-sampling Technique Artificial Intelligence Journal, 16, 321-357. [6] Johnson, R. A., O. Caelen, A. Dal Pozzolo, and G. Bontempi. (2015). Undersampling calibrates probability for the imbalanced classification. IEEE Symposium Series on Computational Intelligence, 2015, 159-166. IEEE, 2013. [7] Han H, Mao BH, Wang WY (2005). Borderline-SMOTE is a new over-sampling strategy for learning from imbalanced datasets. International Conference on Intelligent Computing (2006) 878–887. . Springer. [8] Garcia, E. A., He, H., Bai, Y., Li, S. (2008). ADASYN: An adaptive synthetic sampling approach for imbalanced learning. IEEE International Joint Conference on Neural Networks, 2008, pp. 1322-1328. IEEE [9] Garcia, E. A. and He, H. (2009). Learning from Imbalanced Data IEEE Transactions on Knowledge and Data Engineering, 21 (9), 1263–1284. [10] Jurgovsky, J., Granitzer, M., Ziegler, K., Calabretto, S., Portier, P. E., He-Guelton, L. and Caelen, O. (2018). Credit card fraud detection using sequence classification . Expert Systems with Applications, 100, 234-245. [11] Lemaitre, G., Aridas, C. K. and Nogueira, F. (2017). Imbalanced-learn: A Python library to tackle the challenge of imbalanced datasets in machine learning. Machine Learning Research Journal, 18(17), 1-5. [12] Pedregosa, F., Varoquaux, G., Gramfort, A., et al. 2011. Scikit-learn: Machine learning in Python. Machine Learning Research, 12, 2825-2830. [13] Randhawa K, Loo C K, Seera M, Lim C P, and Nandi A K. (2018). AdaBoost and majority voting are utilized for credit card fraud detection. IEEE Access, 6, 14277-14284. [14] Rehmsmeier, M., and Saito, T. (2015). The precision-recall plot contains more information than the ROC plot for evaluating binary classifiers on unbalanced samples. 10(3): e0118432, PLoS ONE. [15] Machine Learning Group, ULB. (2013). dataset for credit card fraud detection. Kaggle [16] Codefinity: Courses with certificates | Online Learning Platform. (n.d.). Codefinity.

Copyright

Copyright © 2026 Sanket Mohite, Prof. Sunny Nahar. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.